Investigation of a Localized Approach to Shift-Variant Image Restoration and Robust Autofocusing

of the Dissertation Investigation of a Localized Approach to Shift-Variant Image Restoration and Robust Autofocusing by Younsik Kang Doctor of Philosophy in Electrical Engineering Stony Brook University 2011 Images of three-dimensional (3D) scenes or dynamic scenes captured by a digital camera are in general blurred by different degrees at different points in the image. The blur level at a pixel depends on the point spread function (PSF) of the camera near that pixel. The blur may be due to both defocus and motion blur effects. The PSF itself depends on the shape, distance, and motion of objects in the scene as well as imaging parameters of the camera such as focal length, position of image sensor, and camera aperture. Images where the blur PSF changes with spatial position of pixels are said to be shift-variant blurred images. In some special cases such as a planar object perpendicular to the direction of view, the PSF does not change but remains the same at all pixels. In this case the image is said to be a shift-invariant v or convolution blurred image. Image restoration or deblurring is the problem of recovering the focused image from blurred images in the presence of noise. This thesis addresses two closely related problems. One is determining the distance of objects through image defocus analysis for rapid autofocusing of digital cameras. Another is image restoration or recovering the focused image from shift-variant blurred images. Shift-invariant deblurring or deconvolution is a special case of the shift-variant deblurring. Application of this special case to autofocusing of digital cameras is presented in this thesis. In particular, we present a new Robust Depth-from-Defocus (RDFD) method for autofocusing that combines the advantages of both Depth-fromFocus (DFF) and Depth-from-Defocus (DFD). RDFD does not require pixel correspondence between samples and determines object depth by analyzing local energy of image. RDFD is shown to be computationally faster and robust in the presence of small image shifts due to camera motion and hand-shake during image capture. A new polynomial model for Focus Measure (FM) is proposed. This new model approximates actual focus measure as a polynomial of a blur parameter σ. This model can be used to predict the variation of focus measure with camera parameters near the focal position where the blur level is small. Many methods have been proposed to deblur or restore shift-variant blurred images. These methods typically suffer from exorbitant computational cost and memory requirement. Matrix approach is a widely used method for shift-variant image restoration. It formulates shift-variant system as a matrix equation and solves it by inverting a large matrix. Both computational cost and noise sensitivity are serious problems. An N×N image requires inverting an N×N matrix, which is an O(N) vi operation. Therefore computational cost becomes very large even for small images. This method is highly noise sensitive and mere quantiztion noise alone can result in large errors in image restoration. Regularization is needed to reduce noise sensitivity which requires additional computation and complication in the form of selecting optimal regularization technique and parameter. In order to reduce computational cost, some methods use local piecewise shiftinvariance approximation in small image blocks to model shift-variant blurring. Each image block is restored separately through deconvolution in the Fourier domain using the Fast Fourier Transform. The resulting restored image blocks are synthesized to reconstruct the whole image. Due to the inherent approximation used, the error in restoration could be large. In addition, this method could result in highly visible blocking artifacts or discontinuities near the boundary between image blocks. This thesis investigates a new approach to shift-variant image restoration based on a complete localization of the blurring process using Rao Transform (RT). This approach is fully localized, non-iterative, and effective. It assumes a local polynomial model for the focused image, e.g. a bicubic model in a small image region such as 9× 9. The blurred image intensity at each pixel is expressed in terms of the derivatives of the focused image at the same pixel and moments of the derivatives of the shift-variant PSF at the same pixel. This differential equation is inverted locally by considering the spatial derivatives of the blurred image. The resulting solution provides a computationally efficient, parallel, and closed-form solution to the shift-variant image restoration problem. This method is extended by incorporating regularization techniques to derive robust image restoration techniques. In comparison with comvii peting techniques, this new method is fully localized and it involves non-iterative computation in the spatial domain. It enables deblurring to be carried out in the image space and to restrict computations to use local image data. Therefore, this new method permits very fine-grain parallel implementation and is suitable for real-time applications. In this thesis, the RT theory underlying the image restoration method is reviewed, tested, and evaluated in the case of shift-variant blur. The performance of RT for various shift-variant systems including one dimensional blur, two dimensional blur, and multiple blur are investigated. Analysis of noise sensitivity of spatial differentiation filters is presented. A unified expression is derived for multiple blur that includes motion blur and shift-variant defocus. Results of both simulation and real experiments are presented for digital still cameras and cell-phone cameras. Performance of the new localized approach is compared with matrix approaches such as Singular Value Decomposition (SVD) and Conjugate Gradient method. The results indicate that the new approach offers significant computational speed up and improved accuracy in comparison with general Singular Value Decomposition (SVD) and Conjugate Gradient method. In particular, when combined with regularization, the new approach provides computationally efficient and more accurate results.